-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow namespace scoped operator #719
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First round. I am still looking at other files.
pkg/vadmin/at.go
Outdated
@@ -45,11 +46,11 @@ func (a *Admintools) execAdmintools(ctx context.Context, initiatorPod types.Name | |||
// Dump relevant contents of the admintools.conf before and after the | |||
// admintools calls. We do this for PD purposes to see what changes occurred | |||
// in the file. | |||
if a.DevMode { | |||
if opcfg.GetIsDebugLoggingEnabled() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We were doing this in dev mode so why in debug mode now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We used devMode as a proxy for extra debugging. I think its clearer if we use the log level. I should have done this initially.
|
||
# 21. Change change ClusterRoles/ClusterRoleBindings for the manager to be | ||
# Roles/RoleBindings if the operator is scoped to a single namespace. | ||
for f in $TEMPLATE_DIR/verticadb-operator-manager-clusterrolebinding-crb.yaml \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where do you check if the operator is namespace-scoped?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are three places:
- in helm-charts/verticadb-operator/templates/_helpers.tpl, we check to know if a ClusterRole/ClusterRoleBinding should really be a Role/RoleBinding. See the "vdb-op.roleKind" and "vdb-op.roleBindingKing" defines.
- in PodSecurityReconciler, with a namespace scoped operator we don't have permissions to read the annotations of a namespace anymore.
- in pkg/opcfg/config, we use the scope to know what namespace the operator needs to watch
The helm chart parameter `prometheus.expose` controls if the Prometheus metrics in the operator are exposed. The old default was `ExposeWithProxyAuth`. However, this required a ClusterRole/ClusterRoleBinding to be created. This can be onerous, especially in environments where creation of those is prohibited except for cluster admins. The new default is `Disable` to make it easier to deploy in a more restricted environment. This also adds back logging helm chart parameters that were removed in #719. Instead, those parameters have been deprecated and will be removed in a future version.
This change allows for deploying the operator at the namespace scope, only for Helm deployments. When deployed this way, the operator will watch only custom resources within its deployment namespace. Multiple namespace deployments are possible.
By default, the webhook is deployed with the operator. However, with multiple namespace scoped deployments in a cluster, the webhook can only be deployed once.
There are two ways to handle this:
webhook.enable
to false for any subsequent operator deploymentcontrollers.enable
set to false) and deploy each operator with the webhook disabled (webhook.enable
set to false)A new Helm chart parameter,
controllers.scope
, has been added to control the scope of the operator, with valid values of "cluster" or "namespace" and a default of "cluster."The e2e-leg-2 has been converted to deploy the operator with namespace scope.
All command-line options used by the operator have been replaced with environment variables read from a ConfigMap. This change was made when adding the namespace scope and controllers disable options, as this method is more extensible.
This change removes most of the logging Helm chart parameters, leaving only
logging.level
. Those using the logging Helm chart parameters can set appropriate entries in the ConfigMap to retain the old behavior.Finally, the 'make run' option has been removed. The operator can now only be run by running a container within Kubernetes, as it was too difficult to set up a ConfigMap and environment variables to run the operator.